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Integrating Teaching, Advising, and Research Tools: The Student As 
Learner Inventory as Retention and Learning Intervention 



Bernardin Deutsch 

Glen Rogers 
Celestine Schall 
Tamar Ben-Ur 
Debra Chomicka 
Jane R. Frederick 

Abstract 

A locally developed learning inventory — completed by entering and second semester students in an 
undergraduate program and integral to the curriculum — integrates several purposes: 1) self-reflection by 
students on their development as learners, 2) support for student/advisor/instructor discussions on 
learning issues, and 3) identification of students "at risk" for attrition. 

Setting 

Alvemo Cc 'liege is a private four-year liberal arts college for women in Milwaukee, Wisconsin, with 
2,500 degree students enrolled in either a weekday or weekend time fi-ame. Generally, students are from 
southeastern Wisconsin, are first-generation college students, and work during and after college. 
Currently, 72% are Caucasian, 15% are African- American, and 13% are of other cultural backgrounds. 
Overall, 88% of Alvemo incoming students are retained after their first year, with the comparable 
statistic just for nriinority students at 80%. 

Since 1973, graduation from Alvemo has required students to demonstrate eight abilities to an explicit 
level of effectiveness in the context of disciplinary or professional content: communication, analysis, 
problem solving, valuing in decision making, social interaction, global perspectives, effective citizenship, 
and aesthetic responsiveness. Faculty have determined and taught these abilities in general education 
courses and in a variety of disciplines. They have made them explicit through criteria and assessed them 
in multiple modes and contexts through their student performance assessment system. Alvemo has tested 
these abilities by collaborating with many other institutions and their faculty through a variety of 
consortia that cross the educational spectrum from elementary to professional schools (Consortium for 
the Improvement of Teaching, Learning, and Assessment, 1992). 

Introduction 
The Integration of Instruction and Research Purposes 

In our development of an inventory that addresses how students understand their leaming, we have 
constantly held ourselves to the standard of making this inventory meet the immediate teaching and 
leaming purposes of students, instmctors, and advisors. Moreover, we held ourselves to this standard 
before we could hope to have what would be considered a diagnostic inventory that statistically profiled 
the student as leamer. So, we began with the rather simple idea that completing the inventory about 
leaming had to be a leaming experience for the student. 
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All along we have also had a more long-range goal of using the inventory to collect data that would 
enlarge our institutional understanding of the development of students as learners in the College's 
curriculum. However, we have held each administration of the instrument, even those aimed at its 
development as a research tool, to the standard of immediate student learning from the experience of 
completing the inventory. As a result of keeping this requirement for learning paramount, we have 
created a different kind of inventory. This paper is about how the Student As Learner Inventory has 
become integrated into our practice, how it has become a tool for both diagnosing and simultaneously 
intervening around learning issues that may put students "at risk" of attrition, and about how the 
inventory has become a seminal learning experience for students, as well as meeting some of our ' 
research goals. The paper is neither a chronology nor an exhaustive history. In particular, we do not 
review the extensive research that articulated the models. Instead, we try to illustrate the meaning of the 
integration of research and pedagogy in the development and use of the inventory, including its 
application to student retention, by discussing the ideas that emerged from the attempt to accommodate 
these multiple purposes. 

The inventory is different in many ways from most other inventories. As we will show, the way we 
create "scores" is different, and how we interpret them is different. We did not start out looking for 
these differences. We did, however, intend to focus on articulating an institutional understanding of how 
students develop as learners in the curriculum, which we saw as distinctive. And so, we readily 
acknowledge that the inventory embodies a view on learning that is located within a specific curriculum 
(Alvemo College Faculty, 1976, rev. 1985, 1992) and its corresponding educational principles. 

The faculty continue to develop the curriculum and their understanding of teaching, learning, and 
assessment through their research and practice-based experience (Diez, Rickards, & Lake 1994; 
Loacker, 1988; Alvemo College Research and Evaluation Committee, Office of Research and 
Evaluation, and Additional Faculty and Staff, 1993; Riordan, 1993). This reflective practice is also 
informed by institutional and program assessment (Mentkowski, 1994) which broadens research and 
evaluation of the curriculum (Mentkowski, 1988; Mentkowski & Doherty, 1983, 1984; Mentkowski , 
Rogers, Deemer, Ben-Ur, Reisetter, Rickards, & Talbott, 1991). Out of its reflective practice, the 
College has developed a curriculum and a shared perspective on student learning, development, and 
abilities that is distinctively its own (Alvemo College Faculty, 1976, rev. 1985, 1992; Alvem.o College 
Faculty, 1979, rev. 1985, 1994). While the particulars of this shared perspective are the achievements of 
a particular community of educators, many of its underlying principles are widely shared (Consortium 
for the Improvement of Teaching, Learning, and Assessment, 1992). It is beyond the scope of this paper 
to describe fliUy this shared perspective. However, the Student as Leamer Inventory (see Appendix A) is 
itself intended to both reflect and communicate this shared perspective on the development of leamers in 
Alvemo' s ability-based curriculum. 

A Broad Rationale For Research In a Curriculum Context 

Many frameworks are available for understanding student development, learning, and retention. In particular, 
Tinto (1993) has developed a generalizable model of components that affect student retention. Reseaich in 
this model has clarified that students' integration into both the academic and social life of college contributes 
to retention. In Tinto' s model, integration involves the interaction of the college's system, with the individual 
student. Understanding how students become academically integrated into college can be inferred from tiie 
literature on factors that affect student learning and development, for example, the impact of students' level of 
involvement or quality of effort (Astin, 1991; Pace, 1984; Pascarella, 1985). Perry's (1970) work on the 
intellectual and ethical development of college students continues to provide a framework for understanding 
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how college students change their conceptualization of knowledge and learning and how this impacts their 
relationship to college instruction (see Reisetter Hart, Rickards, & Mentkowski, 1995). Kolb's (1984) theory 
of experiential learning likewise offers insight into how different learning styles relate to different instructional 
strategies (see Hutchings & Wutzdorflf, 1988). 

These models of learning, along with Tinto (1993), suggest that academic integration is "a complex 
phenomenon arising fi-om the interaction between student learning characteristics and educational 
practices (see Riordan, 1993). Cross-institution models and instrumentation can provide useful 
fi^meworks and tools. At the same time, institutionally-designed instruments based on the study of a 
college's own student body in the context of the institution's mission and educational philosophy have the 
potential to contribute to a more usable picture of students and their experiences, as well as to the 
development of an institutional perspective on educational principles (Mentkowski, Astin, Ewell, & 
Moran, 1991). 

In the process of developing instrumentation around the learning fi-ameworks of students in the 
curriculum, an institution may have a greater potential for uncovering sources of student difficulty in 
academically integrating with the curriculum. Instrumentation that directs students and faculty 
specifically to fi-ameworks for performance within the curriculum would seem to have the potential to 
more seamlessly contribute to faculty and student understanding of student academic experiences. Thus, 
Rogers (1988) has observed that the contextual validity of assessment instruments and research and 
evaluation practices are critical to their success (cf Wiggins, 1993). In this spirit, the Student As Learner 
Inventory was developed by integrating Alvemo's extensive longitudinal research on student 
perspectives on their learning with faculty deliberation on student development in the curriculum. 

The Contextual Validity of the Original Learning Statements 

Between 1976 and 1980, the college's Office of Research and Evaluation conducted extensive 
longitudinal interviews with a sample of students as they moved through the curriculum (Much & 
Mentkowski, 1982. 1984). It is beyond the scope of this paper to review this research. Nonetheless, the 
research based on these interviews provides a fundamental record of students' own constructions of their 
development as learners. This weights the student perspective in the inventory. With this base of 
research into student constructions in support of a deliberative process, the Assessment Council (a 
committee of senior faculty with responsibility for overseeing assessment on campus) developed a 
portrait of performance characteristics for beginning, developing, and advanced learners. This faculty- 
developed nartative description of the student's development of Six Characteristics of Performance 
(integration, independence, creativity, commitment, awareness of capability, and habituality) further 
elucidates the development of students as learners in the ability-based curriculum. Except to the degree 
that they overiapped with the eight curriculum abilities that students were taught and assessed for 
(communication, analysis, problem solving, valuing in decision making, social interaction, effective 
citizenship, global perspectives, and aesthetic responsiveness) the Six Characteristics of Performance 
were not specifically required of students. Mentkowski and Strait (1983) demonstrated that when faculty 
rated students on these six characteristics a single underlying "developmental" factor accounted for the 
N'ariance in faculty ratings. They found that this ratings-based factor was related to external measures of 
student development. 

Rogers (1988) documents subsequent interactive processes where faculty identified further behavioral 
descriptors of the Six Characteristics. These behavioral descriptors expanded the original item pool and 
sharpened the picture of the developing leainer fi-om the faculty perspective. At the same time, the 
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Office of Research and Evaluation reviewed the analysis of the longitudinal interviews of student 
perspectives on their learning and thereby provided items with language congenial to student 
understandings. This provided an independent analytic framework for describing student development of 
themselves as learners. The Office of Research and Evaluation integrated this analysis with the results of 
the faculty generation of descriptive statements of the Six Characteristics and previous definitions to 
produce multi-item rating forms. The Student as Learner Inventory is a subsequent inventory developed 
for use with students, using these items and conceptual understandings as its base of development. 

Overview of the Student as Learner Inventory 
Purposes, Uses, and Descriiption of the inventory 

Alvemo College developed the Student As Learner Inventory as a multipurpose instrument administered 
to students in the context of their learning in the curriculum. It connects the college's extensive research 
on student constructions of their learning and development (see Mentkowski, 1988) with a strategy for 
students to self-reflect on their performance in the curriculum, while providing data for research at the 
level of the institution. The inventory was introduced for student use in 1986, with major revisions in 
1991 and 1994.' Faculty envisioned and developed the inventory for use within an introductory seminar 
taken by all non-transfer students in the weekday time frame as another way to invite students to self- 
reflect on the ability-based curriculum and the expectations of the college for them as learners. 
Instructors and advisors have continuously used it to support student learning and integration into the 
college since 1986. The instrument measures 
students' adjustment to the academic 
requirements of college and is also an integral 
part of the teaching-learning process. As 
such, it simultaneously informs students, 
teachers, and advisors about learning issues 
and supports conversations with students 
about their role as learners. More recently, 
the capacity of the inventory to both identify 
students "at risk" as learners in the college 
and to support an intervention with students 
that supports their retention has become an 
emergent purpose of the inventory. 

Since 1 986 the inventory has been 
administered in New Student Seminar. This is 
a course taken by all non-transfer students in 
the Weekday College time frame and provides 
an in-depth orientation to college learning. 
Completion of the inventory is integral to 
how students' reflectively integrate their 
learning in the course. As of 1994, the 
inventory is also administered in 
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' The first author, as Coordinator of New Student Seminar, has envisioned and overseen the use of the 
inventoiy as an instructional device. From 1 985 to present, she has been a leader in its conceptualization and 
revision. 
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Preprofessional Seminar. This is a course taken by all non-transfer students in their second semester, 
with the exception of nurses who take a nursing specific version of the course. In this credit course, the 
student develops and explores her interests, her pattern of dealing with others, and her dominant 
personality themes, as well as her learning and work styles.^ The readministration of the Student as 
Learner Inventory offers the student another opportunity to reflect on her learning. 
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Since 1986, the Student as Learner Inventor>' has 
been comprised of sentence stems that elicit a 
self-rating on how characteristic a statement is of 
the student's learning and a self-reflective essay 
on their learning. The inventory advises students 
that "something may be "characteristic of your 
performance" either because you usually do it or 
because it describes your thinking or feeling as 
you perform." The current version has 73 self- 
rating items (see Appendix A). Each item has 

three values "not characteristic" ''partly characteristic" and ''characteristic." When administered in 
Preprofessional Seminar, only reflection topic number ten, "committing to professional goals," is used to 
elicit a self-reflective essay (see Appendix A). 

The ten sections of the inventory present an overview of students as learners at the college (see Figure 1). 
The sequence of categories and the category names, as well as the items themselves, are intended to 
introduce students to a philosophy of how they can develop as lifelong-learners (cf Rogers, 1988). 
The "inventory sections and reflection topics" (see Appendix A) help reinforce for the student a vision of 
how they will develop as independent learners through self-assessment and feedback, so that they 
develop their own perspective, understand multiple perspectives and establish meaningfial career goals. 
The section names (see Figure 1) are not intended to be the basis for the development of subscales. 



College work and independent learning 

— committing to college work 

— becoming independent in learning 

— self-assessment 

— using feedback and assessment criteria 

— commitment to improve 

— understanding Alvemo's integrated learning process 
Personal development 

— developing one's own perspective and being creative 

— applying concepts 

— taking multiple points of view 

— committing to professional goals 



Figure 1: Sections of the Student As Learner Inventory as Pedagogy 
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Through interviews with local professionals, she gathers information that enables her to explore and select a 
chosen field of study. 
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Rationale Supporting the Construction of the inventory 

The Student as Learner Inventory relies on student's constructions of their own learning as the primary 
source of the understanding of learning that it portrays in the item statements. The rationale is that 
students who are "learning to learn" as they enter the college will do that more effectively if they can 
relate to an understanding of learning that reflects how prior students actually experienced and 
understood learning. Thus, the inventory relies primarily on the student's perspective on learning, as it 
developed over time in interaction with a faculty-designed coherent curriculum. Because the inventory is 
based on individual, longitudinal interviews during the college years at Alvemo, it reflects ways of 
learning that move from beginning to advanced understandings within the curriculum. 

The development of the inventory is extended by faculty constructions of learning that mirror the 
educational assumptions and learning principles that ground the curriculum, and therefore, the learning 
experiences that are designed for students across their college years. It also mirrors faculty-developed 
characteristics of student performance, that are not required for graduation, but that faculty teach 
toward in many kinds of ways through the formal curriculum and through their daily interactions with 
students. In that sense, the S .udent as Learner Inventory is an opportunity for students to experience 
faculty constructions side-by-side with student constructions. It organizes the statements or items in 
such a way that the items themselves reflect a perspective on learning and its development that evolved 
from the longitudinal interviews and the faculty's goals for the student as learner. Thus, the inventory is 
another way that the environmental press for learning is communicated to students — by prior students 
and by faculty. 

Collaboration Among Researchers, Instructors, and Advisors 

The Office of Research and Evaluation^ has been the "administrative home" of the inventoi7 and its 
development, with responsibility for coordinating the development and use of the instrument. The 
faculty's Assessment Council'' was particulariy instrumental to the early development of the inventory. 

The ongoing development of the inventory is also a shared responsibility (see Figure 2). Across its nine 
years of use, this development has involved the continuous collaboration of researchers and faculty. 
Figure 2 details the extent of the collaboration among departments and identifies the authors in their 
roles. In addition, the administrative heads of these departments and committees fully supported this 
work.' Many individuals in diverse roles across the campus have substantively contributed. In particular, 
the users of the inventory (course instructors and advisors) have also assumed important roles in 
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Marcia Mentkowski is the Director of this office. She envisioned the development of the student 
constructions as the basis for inventory items and helped guide the early development of the inventory. Oflice 
of Reseai'ch and Evaluation consultant, Nancy Much, provided support. William Rickards has also played an 
instrumental role in developing cross-department conversations around the use of the inventory. 

The Assessment Council is a faculty led committee, chaired by Georgine Loacker. The 1 986 "student-as- 
Icamer subcommittee" reviewed and revised the inventory items (Kathleen Bultman, Wendell Kringcn, 
Georgine Loacker, Marcia Mentkowski, Glen Rogers, and Chnstinc Trimbcrgcr). 
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Other administrators and committee chairs in positions not mentioned above arc: Marlenc Neiscs, (Executive 
Director, Academic Services; co-chair. Retention Committee); Camille Wiersgalla, (past Co-Chair, 
Retention Committee); Elaine Pagliaro, (Co-Chair, Advising Committee); Kelley Talley, (present Co-Chair, 
Retention Committee); Marcia Mentkowski, ( Chair, Research and Evaluation Committee, as well . 
Director, Office of Research and Evaluation) 
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envisioning and developing the inventory. For example, in 1 994, the director of the Academic Advising 
Office, along with the course coordinators, assumed an active role by suggesting alternative response 
formats, revising item stems, and suggesting elimination of items. Each of these activities was then 
negotiated through meetings with research and evaluation staff. For example, suggested revisions of 
items were examined in relation to ( 1 ) the history of the development of the statements and their 
intended meaning, (2) the current research findings, and (3) the broad purposes of the inventory, 
including describing the domain of learning at Alvemo College. 

C \ 

□ academic advising office (Jane Frederick, Director) 

□ advising committee (Jane Frederick, co-chair; Celestine Schall) 

□ assessment council (faculty led, Including research and evaluation representatives) 

□ career development center (Celestine Schall, Director) 

□ new student seminar course coordinator (Bemardin Deutsch) 

□ office of research and evaluation (including Glen Rogers, Tamar Ben-Ur) 

□ pre-professional seminar course coordinator (Debra Chomicka, present; Celestine Schall, past) 

□ retention committee (including Bemardin Deutsch, Glen Rogers) 

Figure 2: Authors in relation to key positions and groups across the institution as they contributed to the 
development of the inventory across the last 10 years 

Reframing Validity As Research and Instruction Are Combined 

As we get to the heart of what we have to say, we return to the modest idea that completing the Student 
As Learner Inventory foremost had to remain a learning experience for the student,* even as we pursued 
our research and diagnostic score-based interpretations of the inventory. The first step we took was 
hardly even an extrapolation of this simple idea. In the first administration, we added a page for students 
to vwite a reflective statement about what they had learned about themselves as a learner by completing 
the invento /. This idea followed naturally fi^om the principles undergirding the philosophy and practice 
of assessment-as-leaming (see Alvemo College Faculty, 1976, revised 1985, 1994). 

By making student learning foremost in the administration and design of the inventory, we were led to 
refi-ame our thinking about validity. Refi-aming validity in this context of student learning meant thinking 
about more than just measurement (cf Moss, 1994; Rogers, 1994). From a measurement perspective, 
validity involves the interpretation of a "score" (Messick, 1993). Often, the discussion of the validity of 
an inventory or test will assume measurement validity is the only meaning for the term validity. And, yet, 
our experience suggests that many contextual factors affect what is a valid interpretative process. 
Bringing a "score" into an interpretative process brings issues of measurement validity, but does not 
thereby exclude other interpretative processes vdth their own "validity" concerns. 

We believe our experience in integrating individualzed instruction and aggregate research purposes has 
taught us something about creating scales and assessing their validity in an instructional context. While 
our practice has engaged some of the murky boundaries that surround the idea of validity, we largely 
confine ourselves to describing some examples of how we have concretely rethought what we were 
doing as we tried to maintain the standard that an inventory about learning should maximize learning as 
it is completed. Before we take up how our research and diagnostic uses of the inventory are beginning 
to be realized, we need to linger on how we first maximized its learning purposes. 



' The Consortium for the Improvement of Teaching, Learning, and Assessment ( 1 992) also ttx)k "what did 
students learn by taking the assessment" as a common evaluative question in the development of student 



performance-based assessments across a vanet\' of campuses 
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Putting Learning First in a Learning Inventory 

Puttiiig learning first in the inveniory has meant that we have made some compromises that affect the 
psychometric properties of the inventory. In this section of the paper we explore some examples of how 
our thinking has been affected by putting learning first and then how we tried to integrate our research 
goals into our design of the learning inventory. 

Inventory as Pedagogy 

First, we have created a simplified response scale, which was a psychometric compromise. 
Psychometrically, we might prefer a response scale with a wide range of values in order to generate 
greater statistical variance. Pedagogically, however, we have strongly preferred a three value response 
scale, because a few distinctive response categories best supports a meaningful discussion of the 
response. We used the following scale, which facilitates discussions about personal stances toward 
learning (see Figure 3). For most items the distribution of responses is strongly skewed, for example less 
than ten percent answering "not characteristic." 

( '. ^ 

Please indicate the degree to which the following is characteristic 

of your performance 

1 Not characteristic 

2 Partly characteristic 

3 Characteristic 

Figure 3: The Response Scale 



Second, putting learning ahead of our research and measurement goals also affected how we presented 
the items on the inventory. Measurement theory generally works on the principle that separate items will 
be completed with independence. Otherwise they take on the undesirable measurement characteristic of 
"correlated error." Thus, psychometrically, we might prefer scattering related items. This is particularly 
important if we are interested in subscales, where method variance — owing to proximity on the 
inventory — might be confused with substantive correlations between items. Pedagogically, however, we 
prefer grouping related items. Such groupings enable the meaning of one item to spill over into the next, 
which is productive for a learner who is trying to develop an overall picture of how all of the learning 
principles and stances relate to one another^. We went even farther, however. Up until 1994, each group 
of items was preceded by commentary that related to students how faculty expect them to develop as 
learners. In effect, we wore helping students see the "socially desirable" response to the items, as part of 
the teaching effort. We stopped doing this only because we developed an alternative strategy. We made 
this commentary about faculty expectations connect instead with the inventory's self-reflective statement 
This self reflective statement is, of course, key to how the inventory becomes a summative learning 
experience. 
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There may even be some psychometric bc" .it from better communicating the meaning of the item 
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Including Research and Measurement Goals 

The examples so far suggest how a pedagogical perspective modifies decision-making in the direction of 
communicating with the students. Accepting leadership from the pedagogical perspective of faculty does 
not mean that their pedagogy is not informed by the perspective of the researcher — or of measurement 
theory. This brings us to a third example of how our thinking was affected by putting learning first. 
Several faculty articulated the perspective that we should explicitly support beginning learners by 
acknowledging that their beginning learning perspective was indeed a legitimate one "at this time." As a 
result, we focuiied on developing a "beginning" student inventory that would both address the 
characteristics of "pre-thriving" learning and positively describe those early learning characteristics that 
students may show even just six to ten weeks into the curriculum.* 

This meant that the most sophisticated learning statements — which were more characteristic of students 
who had developed through prolonged engagement with the curriculum (see Mentkowski & Doherty, 
1984; Mentkowski, 1988) — would only be represented on separate inventories, if at all. Thus, our 
emergent plan was to develop several separate inventories — each inventory with its own version of items. 
Each would describe learning that was consistent with learners at a specific place in the curriculum.' We 
would try to support beginning students with positive statements about them as beginning learners. Of 
course, creating different inventories would mean that the inventory could not measure longitudinal 
change; For this reason, our research and measurement goals would suggest that common items, should 
be given to students at different places in the curriculum. If well designed, such items would be rated 
differently by students with different learning stances. 

Over time, faculty have included in their articulation of a pedagogical stance, perspectives that are also 
compatible with these research and measurement goals. In this new view, faculty put equal emphasis on 
the value of presenting students with a view of how they might develop as learners. This new emphasis 
supports the measurement rationale for using a single inventory with common items for beginning and 
more established students in the curriculum'^. At the same time, we have remained concerned to make 
sure that the items that describe the more sophisticated learning stances do not overburden the inventory 
with a vision of learning that beginning students would not understand. 



' Many students come to the institution with positive attitudes toward learning that enable them to quickly 
construct learning stances that are congruent with the curriculum's expectations. 

' While "developing" and "advanced" student inventories nominally existed as item banks of learning 
statements, they were never administered as separate inventories. 

Indeed, the creation of a positive description of the beginning learner was developed by using many of the 
item-statements that faculty felt described the developing learner. Relatively few descnptions of a beginning 
learner could be construed as positive. In otlier words, if a student was a "beginner" on all statements, she 
was probably more what our faculty would call "pre-beginning." By including the developing statements on 
the pre- 1 99 1 "Beginning Student" version of the inventory, the instrument designers helped to show the 
student a better picture of where her development was expected to lead Finally, the tentative assignment of 
statements to a "beginning," "developing," or "advanced" status has not been tested by use of the inventory 
with students who are more established in the curriculum. The strategies initially used to assign statements 
to a theoretical level of student development as a learner on the campus are descnbed by Rogers ( 1 988). 
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A key to understanding our collaborative success in working across departments and roles is that each 
colleague is ready to take on a pedagogical or research perspective." We are careful to not engender a 
"dichotomous debate" between research and pedagogical implications. And so, the modified pedagogical 
perspective should be understood as an evolving view: research and pedagogical implications are raised at 
key moments in a conversation and accommodated as seems prudent to our college's teaching mission. 

In 1991, we revised the "beginning student" inventory to include statements about learning from more 
"developing" and "advanced" learners in the curriculum. When we made this change toward including 
more sophisticated descriptions of learning on the inventory, we agreed to expand the number of items on 
the inventory temporarily, to develop an empirical basis for item refinement. We also experimentally 
included an alternative item format, which was a forced choice between two statements about the 
characteristics of the student's learning. While we made these temporary research based decisions, which 
strained the attention of the beginning student and the available time for administration of the inventory, 
we did so only because these changes were deemed ultimately compatible with the expanded use of the 
inventory as a teaching and learning device. For example, the goal of having students reflect on changes 
in their learning would be facilitated by having them review their responses across time to the same items. 
Throughout the use and development of the inventory in the courses, we have remained focused on 
teaching and learning as the primary purpose for the inventory. In putting heavier demands on the 
student's attention, we strained — but were careful not to break — the effectiveness of the inventory as a 
device for students to reflect on their learning. 

This brings us to a fourth example of how putting learning first has changed our thinking about the 
inventory. The teaching and learning goals have been sufficient to sustain use of the inventory. Thus, we 
could collect a great deal of preliminary data — without having to focus on effectively reporting data 
summaries. While the inventory has been used as an instructional device in New Student Seminar since 
1986, we did not have a strategy for creating useful scoring procedures for the inventory until 1993. We 
did not report total scores nor subscale scores from the student self-ratings. Since 1986, we occasionally 
reported aggregate frequency data and we analyzed intercorrelations between items, and investigated 
possible differences in how minority students interpret the inventory." Because we essentially had no 
advice on how to interpret the student self-ratings, when we routed the inventory to advisors, we put the 
self-reflective statement on top. 

Even without a score construction strategy, our commitment to making the inventory an effective part of 
student reflection was suflBcient to sustain its use by the New Student Seminar instructors. This success was 
foreshadowed by our 1986 analysis of student evaluations of the inventory as an aid to reflection on their 
learning. We asked students to describe the strengths and weakness of the inventory. The most typical 
strengths were comments such as "it helps me understand myself as a learner" and "it points out questions 
which the student should be asking themselves as they ieam." While the most typical response was to find no 



" For example, research staff have helped write the stimuli for the student's reflective learning statement and 
faculty have consi<5tently emphasized the need for longitudinal data on a single inventory. 

The frequency data has been the most useful of these in developing an understanding of student learning. A 
report in 1 986 noted with some surprise that half of the students in New Student Seminar acknowledge 
some "emotional reaction to feedback that interferes with insight into her performance." An early Factor 
Analysis in 1 987 of 30 items, with an n of 1 84 . yielded three possible Factors, but they accounted for only 
1 8% of the variance. In 1 99 1 , a faculty expert in culturally based learning differences reviewed the 
inventory for any cultural biases: Peter Murrell (personal communication, 1 99 1 ) reported that the inventory 
did not appear to have any obvious cultural biases and, moreover, that the inventory's s>'sternatic focus on 
learning was a critical means for productively addressing potential learning and cultural differences. 
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weaknesses, some students did make observations like "some of the questions were vague to me" and "too 
many questions," both of which could be seen as valid complaints at that time. Thus, the sustained use of the 
inventory for eight years was based on the eSectiveness of the inventory as a learning tool. We had some hopes 
for it as a research tool, but this latent promise of the inventory remained unrealized. 



Integrating Research and Measurement with Instruction 

While we were glad the inventory was meeting its first priority of promoting learning, we were concerned 
that instructors and advisors were reporting that the length of the inventory made it difficult for them to 
find a focus in their review of students' responses. As we addressed the question of how we might 
facilitate interpretation of the inventories, we were struck not only by the psychometric limitations of the 
self-ratings, but also by the difficulty of communicating tentative and developing understandings. Also, 
we confronted real logistical hurdles if we were to get the inventories scored and routed to instructors 
and advisors when they would be meeting with students. 

Reframing Validity In An Educational Context 

Ultimately, we built our strategy for facilitating interpretation of the self-ratings to be congruent with 
what our faculty and advisors were already doing in their use of the self-ratings. They were interpreting 
the student self-ratings in the context of other information about the students. The scoring and 
interpretation innovation we ultimately developed integrates teaching, advising, and research fi-ameworks. 



Focusing on Specific Responses 

We realized we needed to facilitate an interpretation of student self-ratings so that they could be tied to 
the individualized analysis of the qualitative data on the inventory, as well as to the other data on the 
student that the instructor and advisor were integrating in their interpretative analysis (See Appendix B). 
This scoring interpretation needed to be made in the real-time of instruction and advising and needed to 
include our research-based understandings. The "real-time" interpretation strategy involved, in part, 
focusing attention on very specific responses. In the sample item below, only the first response is 
designated as potentially incongruent with successful academic integration. Thus, for designated "pre- 
thriving" items one specific "incongruent" value is bolded (see example below) on the carbon copies of 
the inventory that are used by the instructor and advisor (see Appendix A). 



E.g., item stem. "I do more than asked for' 

1. "not characteristic of my performance" 

2. "partly characteristic of my performance" 

3. "characteristic of my performance" 



Through the "at a glance" holding of responses, we support a real-time interpretive reading of the seif- 
ratings, and a real-time intervention. A key to our success in enabling utilization of the inventory is that 
advisors and instructors do not need to wait for a report based on data entry. A bolded carbon copy of 
the completed inventory is immediately routed to the student's advisor, and one is available to the 
instructor as well. In addition, the hand "scoring" is so effortless, that it hardly even seems to be a 
"scoring" process. 
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By holding selected response values, instructors 
and advisors notice potential "at risk" responses "at 
a glance." Without the holding, the reverse scored 
items create perceptual barriers to distinguishing the 
"at risk" responses "at a glance " An additional 
perceptual barrier to reading the inventory, without 
the selected holding, is that those "low" responses 
that are "typical" of thriving beginning student need 
to be distinguished from those that are "at risk" 
responses. For example, while we would like 
students to come to say "I enjoy exploring assessor 
responses in relation to my own analysis of my 
performance," a beginning student may not have 
enough experience with our system of assessment 
and learning to develop this confident attitude. 
Thus, their response, "not characteristic of my performance" might be little cause for immediate; concern. 
Unpacking the implications of how we focus on specific infrequent responses illustrates how our instructional 
and aggregate research purposes are integrated within the educational context of the curriculum and how this 
integration has led us to reframe our approach to validity. 

Conceptualizing the Pre-Thriving Construct 

An explicit goal of New Student Seminar includes helping the students construct and continue to develop 
a viable perspective on their learning in the college and afterward. Thus, the inventory would be expected 
to "makes sense" to many students only as they are informed about and begin to experience the ability- 
based curriculum. The instrument addresses the degree to which the students are able to construct an 
initial image of themselves as self-directed learners that is congruent with fundamental principles that 
undergird the college's expectations for them as learners. There are intensive efforts to help students 
construct this congruent identity as learners, so the inventory really addresses a facilitated achievement of 
academic integration in a context where these educational principles have been articulated (Alvemo 
College Faculty, 1976, revised 1985, 1992; 1976, revised 1985, 1994). In this context, the "at risk" 
response is conceptually defined as a rejection of a key principle that the student will need to embrace as 
a thriving member of a community of learners in a coherent curriculum. 

Because the integration of beginning students into the academic milieu of the college generally proceeds 
along a well-supported path, only a few students show a tendency to reject the particulars of the Alvemo 
philosophy of learning. This lack of statistical variance, of course, puts a strain on traditional strategies 
for developing a measure that is supported by item-level analysis. Appendix C shows the frequencies of 
these responses in the form of a handout used to inform instructors and advisors of the distribution of "at 
risk" (or "pre-thriving") responses on the 1991 version of the inventory. In the college we use the term 
"pre-thriving responses" to highlight — for those "at risk" — the potential to grow into thriving students. In 
Appendix C, the "pre-thriving" responses are sorted by frequency and by the direction of response 
scoring. The potential lack of integration into the curriculum is signaled, perhaps over-dramatically, by 
labeling potentially "pre-thriving" responses as "Not Me," or "That's Me," depending on the direction of 
the response that is scored as "at risk." The hand-out also specifies infrequent (but not rare) responses 
that are associated with student attrition at the level of the item. Overall, the handout is distinguished by 
the fact that it draws attention to the rarest and most infrequent responses students make. Most of the 
items have less than five percent of the students giving a "pre-thriving" response, and so no item level 
correlation is given for these. 



Context sensitive and real-time interpretation 
♦ Faculty and advisors use such bolded 
responses in relation to the student self- 
reflection statement, and other 
knowledge of the student's performance, 
to discuss potential learning issues wth 
thestudent;::;- ■,/■ 
•.Score "iiirterpretation' in tiiebontextof a 
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The effective integration of the 
aggregate research and instructional 
purposes meant embracing a 
conceptuat scate consU^uction 
framewof1< 



Once we realized that the rare and infrequent rejection of a key Alvemo learning principle could be 
summed across items, and committed ourselves to this theoretical stance, we then could develop a real- 
time interpretation strategy," Our key insight was that the "rare" response could have an intrinsic 
theoretical meaning. In particular, we theorized that each individual student who was not integrating into 
the educational philosophy and principles of the college had a relatively different set of principles that 
they had difficulty accommodating. First, students might have specific "gaps" in their understanding of 
the college's learning principles. Second, even if they 
understood the learning principles, they might reject 
some. In particular, students with different world views 
nrught create different "defenses" for the traditional 
learning expectations they brought with them, such as the 
expectation for grades. Thus, fairly infrequent ways of 
rejecting or misunderstanding the educational perspective 
underlying the curriculum — as well as some more 
common learning issues — became theoretically important. 

The more non-normative the response, the more likely we might be identifying a unique way the student 
was constructing a maladaptive "rejection" of the expectations of the Alvemo student as learner. 
However, if we were to pursue this theory, we had to give up on even the possibility of an empirically 
based item-level analysis of the "at-risk" scale we constructed. While developing a theoretical rationale 
for how to construct a scale is congruent with the highest form of psychometric judgment (cf Loevinger 
& Wessier, 1970), the impracticality of an item-level empirical test of the scale construction did 
temporarily blind our research perspective to this kind of a solution. 

Statistically Validating the Pre-Thriving sscale 

Although we could not statistically conduct systematic item analyses, we could test whether the number 
of "pre-thriving" (at risk) responses was related to retention. So, we tested this hypothesis before we 
asked instructors and advisors to try out the facilitated and real-time interpretation of the "at risk" 
responses.'* But, as a test, we were not satisfied with just a test of whether the inventory predicted 
retention. After all, some students are admitted to college under "academic status" advisement based on 
their prior high school or college work. At the end of each semester, the college's Academic Status 
Committee reviews the performance of each student who had academic difficulties that semester or who 
was already under "academic status" adxasement. In addition, the college has an extensive pre-college 
program and instructional services (Alvemo College, 1993-94)'that potentially can provide early 
information on students that may confront academic difficulties. So, we regressed retention'' in sequence 
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A spur to making this theoretical analysis was tlie observation that for most items "partly characteristic " 
seemed to be a potentially sophisticated response. In particular, as students develop self-awareness about a 
learning issue, and see better the range of ways to develop a yet more articulated and consistent stance, they 
may be more likely to share a self-rating that reflects their understanding of limitations. 

Our analyses are based on data from 3 1 3 women who in the Fall of 1 99 1 or 1 992 completed the 1 99 1 
version of the Student as Learner Inventory in New Student Seminar. New Student Seminar is a course for 
all entering non-transfer students in the Weekday College time frame. 

For the purposes of these analyses, one-year retention is defmed as a student completing their first two 
semesters. Technically, if the College's computer records indicated they had an attrition date code of April, 
which is near the end of the second semester, we assumed they completed that semester. Two year retention 
is correspondingly defmed in April of the fourth semester. Retention information was missing for one 
individual. 
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first on to academic status'* and then on to the number of "pre-thriving" responses on the inventory to 
test whether the number of "pre-thriving" responses predicted retention above and beyond first semester 
academic status. After the forced entry of academic status into the model, the number of "pre-thriving" 
responses still predicted one year retention, 6(1, 309) = -.24, p < .001), and two year retention 6(1,309) 
= p< .01)." 

In 1993, faculty and staff representing diverse interests in the inventory reviewed these results and critiqued the 
educator's guide to the inventory developed around the "pre-thriving" item concept. While the above data 
justified our efforts to develop a facilitated interpretation of the "pre-thriving responses," the strategy of 
focusing on specific values of specific items does potentially overlook valuable information. 

Benchmarking Against Alternative Scale Construction Strategies 

To test how well our facilitated interpretation strategy utilized the retention relevant information in the 
self-ratings, we compared the "number of pre-thriving responses" index that we created with two 
alternative indices. One alternative, which we entitled the "conventional pre-thriving scale total," was 
comprised of the same item stems, but included all three scale points for each item, with a traditional 
summation across the items. The other alternative maximized not only the use of each scale point, but 
also maximized the number of items. To contrast with the 42 items that focused on basic expectations for 
beginning learners in the college, this alternative scale used all 82 items on the inventory. 

Table 1 displays side by side the observed capacity of the three scales to predict one- and two-year retention. 
The three alternative scales fi-om the Student As Learner Inventory show a relatively similar predictive 
relationship to student retention (see Table 1). The slightly reduced n for the "conventional pre-thriving scale 
total" relative to the index based on "number of pre-thriving responses" arises fi-om differences in how we 
treated missing data. In particular, for the index based on the "number of pre-thriving responses," we have been 
able to make the simplifying assumption that a missing response can be assigned the value of zero (that is, not 
pre-thriving) since pre-thriving responses are rare or very infi-equent.'* 

Table 2 displays side by side the observed capacity of the three scales to predict academic status at the end of 
two years in the curriculum. The three alternative scales fi-om the Student as Learner Inventory have relatively 
similar predictive relationship to academic status two years later (see Table 2). Thus, we have evidence that the 
facilitated interpretation strategy is equally efficient in using the available data from the inventory. Although 
small, the correlations with retention and academic status displayed in Tables 1 and 2 provide some statistical 
evidence for the construct validity of the pre-thriving scale. Given the use of the inventory to intervene around 
learning issues, we cannot assume that higher predictive correlations should be expected. Also, of course, 
academic integration is only one of many factors in retention. 
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For the purposes of this and oilier analyses using academic status as a variable the following values were 
assigned; Good standing = 0, special advisement = 2, probation = 4, probation with warning = 6, and 
academic dismissal = 8. At entrance valid values are 0, 2, and 6. Cross-tabulation of the academic status 
scale with the inventor> 's "pre-thriving" scale corroborates the linear nature of the academic status scale, at 
entrance and at 2 years. (Dctjioocm 

Data printout ID# (DC SLOOO 17) 



" The development of mean substitution scoring program for missing data on the conventional total scales is 
technically feasible, but we have not yet established it as a priority. 
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Contextual Features to the Valid Interpretation of Scores 

Our strategy of facilitating an interpretation of self-ratings— in comparison to an isolated reported 
score — has another advantage, aside from being done in real-time. By placing the interpretation of the 
student rating next to the student's self-reflective statement on the inventory — as well as other qualitative 
data that instructors have at hand — and by placing the interpretation next to an opportunity to discuss it 
with the student, we have created a sensitive context that mitigates over-interpretation of the self-ratings. 
It should be noted that the Standards for Educational and Psychological Testing emphasize that 
individual items, are subject to considerable "error;" and in their interpretative use, advises a provisional 
interpretation that leads to further enquiry (American Educational Research Association, American 
Psychological Association, National Council on Measurement, 1985). 

Table 1 : Correlations of Student as a Learner Inventory With One-Year and Two-Year 
retention By Scale Construction Strategy 



Scale Construction Strategy 


n One-Year Retention 


r 

Two-Year Retention 


Entered In The Fall of 1991 


Number of Pre-Thriving Responses' 


198 




-.29*** 


Conventionally Scored Sub-Scale" 


173 


.30*** 




Conventional Total Inventory Scale'^ 


153 


.26** 


.25** 


Entered In The Fall of 1992 


Number of Pre-Thriving Responses 


114 


-.24* 


-.11 


Conventionally Scored Sub-Scale 


96 


.28** 


.18 


Conventional Total Inventory Scale 


91 


.31** 


.16 



Note. The analysis is based on the 1991 version of the inventory. 

* The "number of pre-thriving responses" is based on a response to a particular value of the scale, 
either 'not characteristic" or "characteristic". A higher number of pre-thriving responses indicates 
less integration into the curriculum learning expectations. This scale is based on 42 of the 82 
inventory items. 

" The "conventional scoring" of the pre-thriving scale is the simple total of the above mentioned 42- 
item scale. A higher score indicates a higher level of integration vwth the curriculum learning 
expectations. 

' The "conventional inventory scale total" is the simple sum of the 82 inventory items. A higher score 
indicates higher integration^' 



In an ability-based curriculum that relies on performance assessments, faculty and advisors are accustomed to 
making judgments from qualitative data. In panicular, instructors are experienced in making judgments of 
performance that are based on explicit performance criteria — where part of the assessment-as-leaming process 
entails giving feedback to students on their performance on these criteria, as well as making an interpretation of 
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the student's performance that is informed by the student's self-assessment (Alvemo College Faculty, 1976, 
revised 1985, 1994). In this context, instructors and advisors are unlikely to use isolated scores from 
instruments and, instead, expect that their interpretations will be made in the context of a range of information 
about performance. Thus, there is very little pressure to over-interpret "scores" when a wealth of data sources 
and personal observations are available for students. 



Table 2: Correlations of Student as a Learner Inventory With Academic Status at the End of 
Second School Year By Scale Construction Strategy 



Academic Status" at the End of 2nd Year 


Scale Constnx:tion Strategy 


n 


r 


Entered In The Fall of 1991 


Number of Pre-Thriving Responses' 


202 


.31*** 


Conventional Pre-Thriving Scale Total'' 


176 


-.31*** 


Conventional Inventory Scale Total'' 


156 


-.27** 


Entered In The Fall of 1992 


Number of Pre-Thriving Responses 


115 


.33*** 


Conventional Pre-Thriving Scale Total 


96 


-.36*** 


Conventional Total Scale 


91 


-.36*** 



Note. The analysis is based on the 1991 version of the inventory. 

' The "number of pre-thriving responses" is based on a response to a particular value of the 
scale, either 'not characteristic" or 'characteristicf . A higher number of pre-thriving 
responses indicates less integration into the cuniculum ieaming expectations. This scale is 
based on 42 of the 82 inventory items. 

" The "conventional" scoring of the pre-thriving scale is the simple total of the above 
mentioned 42-ftem scale. A h^her score indicates a higher level of integration with the 
curriculum learning expectations. 

' The "conventional inventory scale total" is the simple sum of the 82 inventory items. A higher 
score indicates higher integration 

" Academic status ranges from 0 (in Good StancSng) to 8 (Academic Dismissal) 



A key safeguard against over-interpretation of the inventory is that the student is brought into the 
interpretation process. Indeed, instructors and advisors use the self-ratings to support a conversation with 
the student, where the student provides her own perspective on her self-rating. For example, a student 
might simply explain that a particular response was a "mistake." The discussion of any implications for 
how the student's learning perspective is or is not conducive to learning in the curriculum is taken up as 
just another conversation, like any other, with her inventory responses as one source of grist to that 
conversation. Thus, the completed Student as Learner Inventory, is one of many sources of information 
that are included in the student's "Academic and Career Resource Journal," which she herself is 
responsible for maintaining. The primary use of this student portfolio is as a source of student self- 
reflection. The student copy does not have "pre-thnving" responses bolded, however, and the student 
does not see nor discuss "the number of pre-thriving responses." This helps avoid accidentally 
miscommunicating to the student that a "score" implies some kind of a label. 
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By having an advisor or instructor "score" student self-ratings in the context of other qualitative 
information and a student/advisor discussion we have changed the whole character of the interpretation. 
It has become provisional, negotiated, and contextualized (cf Moss, 1994). As a result, the interpretation 
is something new. It is more than just one more notch up the "pre-thriving" scale. It is a particularized 
item, blended into a wider interpretative judgment process. In our best practice any interpretation is 
double checked against further discussions with the student as part of an attempt to help the student make 
meaning out of the curriculum. 

Some Measurement Perspectives on Score Uses 

Because we have, however provisionally, created "scores" with unknown reliability and many unknown 
measurement properties, we are committed to the continued investigation of these measurement 
properties and the continued contextualization of these scores in an interpretation process. Indeed, 
analysis of one-year retention for the Fall 1993 New Student Seminar cohort did not yield statistically 
significant correlations with the number of "pre-thriving" responses, r (145) = -.03. One possibility we 
have considered is that the interventions of instructors and advisors in relation to the inventory may have 
been so effective that they have made the relationship between pre-thriving responses and retention go 
away. The relationship between the number of "pre-thriving" responses and academic status did persist, 
r(.147) = .35,p<.001.'° 

Because the inventory has maximized learning purposes over measurement properties, we intend to 
maintain the inventory as a diagnostic and research tool. We do not plan to use it for student placement 
or performance evaluation. Thus, we have considered measurement validity to be an important 
consideration in determining the appropriate use of the inventory, even as we reframe the meaning of 
validity in our instructional context.^* 

items Specific to Local Context and Philosophy of Learning 

As is clear from this analysis, some of the learning issues that are key to the diagnosis of congruence with 
the institution's expectations for the active learner are going to be specific or unique to the local context, 
in particular, to the agreed upon learning principles that are part of the college's ability-based curriculum. 



^° A related issue is how the number of pre-thriving responses relates to factors, such as minority stattis, that 
are of particular concern when developing an inventory. The size of the sample only enabled comparisons 
between 48 African-American students and 229 Caucasian students. The number of pre-thriving responses 
was slightly related to African-American background in this sample, r(277) = -. 1 4, /? < .05. When the 
African-American students were included in a separate analysis, the relationship of the number of pre- 
thriving responses to retention was about the same, r (48) = -.22, p> .\\. Although this correlation did not 
reach statistical significance, this could be attributed to the size of the sample. Our interpretation is that the 
inventory is equally effective in assessing integration into curriculum for students of different cultural 
backgrounds. Also (see page 10, footnote 12 above). .su«, 

^' As has been noted, the educators' guide (see Appendix B), significantly contextualizes the score 
interpretation process, which is contemporaneous with the students completion of the inventory. On an 
exploratory basis, we have also shared with the Advising Office and Course Coordinators a list of students 
who meet the cut-point number of "pro-thriving" responses. When using this report, wc strive to maintain 
as many of the contextual characteristics of the real-time interpretations as possible. For example, when a 
list of students who potentially have "pre-thriving responses" in New Student Seminar is forwarded to the 
next instructor in Preprofessional Seminar, the report identifies the specific pre-thriving responses and is 
attached to a copy of the student's completed inventory. Advisors have to consult tlie copy of these 
responses that was routed to them previously. In this manner, they also recontextualize the listed responses 
^ in an overall interpretation of the inventory and its reflective statement 
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Many of the inventory's items (but not all) are specifically situated in the college's philosophy of learning. 
For example, Alvemo's ability-based curriculum is supported by an articulation of how students develop 
self-assessment. The curriculum emphasizes the student's development of her self-assessment of abilities. 
She learns to self-assess what abilities she has demonstrated in performance relative to explicit criteria, 
which she internalizes (Loacker & Jensen, 1988). Thus, the following inventory item reflects a key 
learning concern in the ability-based curriculum. 

— "I review my work, using criteria as a basis:" 

Another item addresses a learning issue that arises in a non-graded assessment sy.stem. Students are 
expected to demonstrate the Alvemo abilities to explicit levels of performance. Feedback is in terms of 
what criteria are met and the level of performance in relation to these. Students do not receive grades nor 
explicit feedback on their performance relative to their peers. Instead, they are expected to take hold of 
their learning through a different kind of self-assessment comparison, trying to improve their performance 
over time relative to performance criteria (Alvemo College Faculty, 1976, revised 1985, 1994). So, the 
following item represents a perspective, crucial to integration into the non-graded system, that faculty 
explicitly try to help students embrace: 

— "I compare myself to myself rather than just myself to others." 

Revision Based on Statistical Techniques and Informed Judgment 

One key to the development of the inventory has been the decision to give instructors a key role in the 
revision of inventory items and the structure of the inventory in light of the course context. Their direct 
experience with the student population and with the use of the inventory gives them a critical perspective 
for evaluating the inventory. At the same time, analysis of longitudinal Perspectives Interviews 
(Mentkowski, 1988; Much & Mentkowski, 1984), statistical analysis of the relationship among items, and 
the concept of "pre-thriving" responses were key bases for making judgments about items. 

The development of the real-time interpretation strategy and evidence of the relationship of "pre-thriving" 
responses to retention together sparked renewed interest in expanding the use of the inventory. This 
renewed interest helped create the context for significant involvement of the users in the revision of the 
inventory. Indeed, the coordinator of the Preprofessional Seminar^^ noted the need to streamline the 
inventory so that it could be feasibly administered in Preprofessional Seminar. As a result, an instrument 
revision team"^^ over the summer of 1994 worked with researchers to revise the instrument and articulate 
its uses in this course context. The readministration in this second semester course has brought us closer 
to our long-term goal of collecting longitudinal data on the inventor>' as a natural outgrowth of its use in 
the curriculum.^* The readministration has also brought us closer to the pedagogical goal of enabling 
students to reflect on changes in their perspective on learning.^' 



" This second semester seminar is a developmental series of activities designed to focus students on the 
process of identifying career options in relation to their individual interests, values, and academic 
ambitions. 

" This team included Debra Chomicka (Coordinator of Preprofessional Seminar), Jane Frederick (Director of 
the Academic Advising Office), and two members of the Office of Research and Evaluation (William 
Rickards and Glen Rogers), Bemardin Deutsch (Coordinator of New Student Seminar) also contribut' i 

" Next semester will be the first time the same inventory is administered in these sequential courses. 

We wonder whether one semester will be enough to reveal these changes. Ultimately, we hope to use the 
mvcntorv at later points m the cumculum as well. 
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Figure 4 displays the range of considerations used by the revision team to reduce the number of item 
statements from 104 to 73. These include the statistical performance of the items in relation to each other and 
to retention, their qualitative contribution to a complete portrait of student learning, and other considerations — 
such as clarity — that connect to the instructors' and advisors' discussions with students around the items. The 
process of changing items from third person to first person, by itself, brought a new perspective to the quality 
of the item statements. Thus, the revised inventory and items were intended to meet the requirements of 
usability, breadth of coverage, faithfulness to the research-based meaning implied in the original item pool, and 
new learning about those meanings both from experience and research. 

C \ 

• Clarity of language for beginning student 

• Helps communicate development as a learner In a specific and critical area 

• Meaningful to beginning students 

— more complex Items paired with similar less complex Items 

— student and faculty perspectives on learning represented 

• In context, projects a positive view of students 

— "negative" Items are retained only if used by students 

• Might Identify those most 'at risk" as learners 

— conceptually designated as "pre-thrlvlng" Item 

— Item level correlation with retention (limited help) 

• Has a meaningful loading In exploratory factor analysis 

Figure 4: Criteria for retaining or rewriting items 



Perhaps one of the most conceptually challenging measurement concerns was consideration of the 
structure of the inventory. On the one hand, we wanted to keep items based on their use in the "pre- 
thriving" scale and its conceptual scale construction strategy. This strategy assumes they cohere as one 
"pre-thriving" construct. On the other hand, we wanted to simultaneously pursue alternative st ale 
construction strategies that might reveal some subscales within this domain of integrating into the 
curriculum. To this end, we conducted a factor analysis of the 1991 version of the inventory. This 
analysis used the fiill set of three response values. Although this analysis hinted at some structure in the 
inventory, it did not yield particularly clean factors, which was not surprising, given the limited variance 
in the response scale^*. Nonetheless, we used this factor analysis as one frame for selecting items to keep. 
This strategy both held open the possibility of identifying different kinds of learning issues in subscales 
and ensured that the inventory would include a multifaceted coverage of the student constructions of the 
curriculum's learning principles". 



" The Principle components factor analysis («=460) was conducted iteratively. Items with low 

"commonality" were excluded, leaving 52 of the 82 items on the 1991 inventory in the analysis. The 
resulting five factor solution accounted for 27 percent of the variance. Of the 52 items, 40 had a loading of 
at least .30 — with all off-diagonal loadings smaller by at least . 1 0. This was used as a cut-off for selecting 
items that would be of interest for a factor approach to inventory development. Of these 40 items, however, 
only 1 5 did not have an off-diagonal loading of .20 or more for one of the other factors. 

In order, the five factors were ( 1 ) Conceptual Involvement in Learning (e.g., doing more than expected, 
affirms relation between theory and practice), (2) Career-Oriented Learning (e.g., relates career goals to 
educational experience), (3) Ability-Based Learning (e.g., use of feedback), (4) Directed Learning (e.g., 
following directions), and (5) Commitment to Academic Work (e.g., shows concern about personal life that 
keep her from achieving). The number of items targeted from each factor as having a factor-based interest 
for inclusion m the revised inventory were, respectively, 13 ^,7, 7, and 4. Because this was a secondary 
cnterion for consideration, only roughly eighty percent of tl. e items were kept for the 1 994 version of the 
inventory 

Er|c 24 



Integratng Teaching.^dvising, and Research Tools 



20 



Summary and Conclusion 

Because the inventory is embedded in a coherent curriculum with a distinctive learning philosophy, we do 
not expect the inventory itself to be transportable to another institution. In fact, we believe the 
development of the inventory for a specific context is the basis for its strength and this is something we 
advocate. Nonetheless, we do believe that our experience in integrating research and instructional 
purposes in the inventory may be of interest to others. The principles that undergird the inventory are also 
of common concern. While our research achievements with the inventory are modest so far, we have 
made some conceptual breakthroughs that give us great expectations. We present one overarching lesson. 
Our efforts toward prioritizing student learning from the inventory — even as we developed and fielded 
it — has put us in the position for simultaneously achieving our research, teaching, and advising goals. 
Putting learning first meant using a response scale with a few distinct alternatives, organizing items 
pedagogically, including a self-reflective essay, embedding the inventory in an opportunity for a 
discussion about learning, and creating items that positively communicate student constructions on 
learning. The inventory is an integral part of instruction, not an externally administered instrument. 

Because we have had to look for opportunities that meet our latent research and scale construction goals, 
we have developed a different kind of strategy for constructing scale scores and for facilitating 
interpretation of the student self-ratings. In 1993, we developed a scale and a real time strategy for 
instructors and advisors to identify students with "pre-thriving learning stances." The "pre-thriving" scale 
identifies responses that are unusual and that suggest intervention. For example, a student indicating that 
she does not "try out new skills in new situations" can be approached about her reasons for this response. 
Thus, teachers and advisors are encouraged to use the student endorsement of specific item responses to 
foster a unique dialogue with students. By focusing on infrequent responses, the scale construction of the 
inventory is based on a congruent principle that individuals may have relatively unique difficulties. For 
example, moving into a non-graded curriculum means most students have to construct new learning 
stances, and students with difficulty in integrating into the curriculum may have constructed relatively 
unique "holes" in their understandings that prevent them from thriving as learners. 

Our scale construction strategy has been conceptually based. Extensive research and faculty experience has 
provided a basis for identifying a wide range of learning issues and student perspectives where "pre-thriving" 
learning stances might develop. A corresponding key role of the inventory is covering a broad range of 
potential learning issues. For example, instructors and advisors report a prior awareness of many of the issues 
that become apparent in student responses on the inventory, but also note that new issues emerge. 

The integration of "pre-thriving scores" into the instructors and advisors ongoing interpretative processes 
has led us to reframe our validity concerns. Our interpretative practice is informed by both learning and 
measurement perspectives on validity. The integration of the inventory into a teaching and learning 
process contextualizes ongoing interpretations. Focusing on specific responses helped us integrate 
teaching, learning, advising, and research purposes. 

Aggregate findings help inform the use of the inventory by the teachers and help validate the construction 
of an overall scale score. "At risk" responses have been correlated with student academic status in the 
first two semesters. The correlations with one and two-year retention have been less consistent across 
these years, but are in the expected direction. The psychometric "limitations" of the inventory — for 
example 37 percent of students had ,io "pre-thriving" responses and only 18 percent had four or more 
such responses — are counterbalanced by the effectiveness of the inte§i;ation of the inventory into the 
teaching and learning process. ^ 0 
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Collaboration among individuals filling diverse roles and departments across the institution has been key 
to the development and effectiveness of -the inventory. Key among these has been the ongoing discussions 
among instructors, advisors, and researchers. Collaboration is a result of common commitments to 
developing smdents as learners and of the hard work of communicating understandings. This has meant 
arranging opportunities to discuss issues — with individuals fi-om each role taking on both teaching and 
research issues as part of their responsibility. One crucial step in this process has been the development of 
the "Educator's Guide to the Student As Learner Inventory," which provides not only the "scoring" and 
interpretation advice, but also describes the context of the administration and other possible uses (see 
Appendix B). 

The integration of teaching, learning, advising, and research purposes means that students benefit directly 
and indirectly fi-om completing the inventory. They benefit directly by learning about what it means to 
integrate into the curriculum. In New Student Seminar, completing the inventory is an integral part of the 
course's summative experience. Students learn about key educational principles and are apprised of how 
they are begirning to integrate into the curriculum. The inventory provides students the opportunity to 
share a range of their views on learning. This includes a crucial self-reflective statement by students. 
Thus, teachers are able to bring them into a conversation about their learning perspective in relation to 
what will make them effective as learners in the curriculum. The inventory not only helps teachers and 
advisors identify students who have learning issues that put them at risk, but it also helps them construct 
a timely intervention. The inventory does more than predict students who may be at risk of leaving the 
institution, it helps do something about it. Finally students indirectly benefit. The inventory supports the 
college's research into the development of students as learners in the curriculum. The longitudinal 
administration of the inventory simultaneously supports the student's own self-reflection on her 
development as a learner in the curriculum. She has another opportunity to take hold of her learning. This 
is one of the highest goals we have as educators. 
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Student as Learner Inventory 



ALVERNO COLLEGE 
Milwaukee, Wisconsin 



Learning at Alvemo may be different from other experiences you have had. By completing this 
inventory, you have an opportunity to broadly reflect on your development as a learner. The last page of 
the inventory invites you to write a reflective statement about your approach to learning. 

Instructions for Completing the Inventory 

Most of the pages of this inventory consist of different statements regarding the behaviors which students 
display as they develop as learners. All statements are legitimate descriptions of students at various 
points of development. Read each statement, and then indicate the degree to which it is characteristic of 
your current performance as a learner. Please, indicate your judgments by circling one number, as 
shown in the example below. 



Example: 



Circle One 
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40. I actively build on strengths. 



Note: Something may be "characteristic of your performance" either because you usually do it or 
because it describes your thinking or feeling as you peiform. Sometimes you may want to distinguish 
between these "characteristics of your performance" and "who you are as a person." 
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INSTRUCTOR C^^ — » RESEARCH 



CAREER DEVELOPMENT 



STUDENTS mME (Please Print) 


ADVISOR'S NAME 


STUDENT ID NUMBER 


DATE 


CLASS 



COLLEGE WORK AND INDEPENDENT LEARNING 
Committing to College Work 

At this time, do the following characterize (describe) my performance? 

1 . I follow concrete suggestions, explanations, and directions in my learning process . . 

2. I do more than expected 



3. I attend class regularly 

4. I am enthusiastic about my coursework 



5. I respond to situations in which I am personally asked to participate 

6. I do more than asked for 



7. My concerns about my personal life keep me from achieving in my academic work . 

8. I complete assignments on time 



9. I participate in class activity 

10. I set goals and work towards them 



Circle One 
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Becoming Independent in Learning 

Do the following characterize (describe) how I am learning this semester? 

11. I accept personal responsibility for learning when asked to 

12. I see tiie teacher as an organizer of learning opportunities 



13. I try a new approach to learning or problem solving when it has been demonstrated 

14. I select an appropriate framework for analyzing a work (something someone created), 

even wdien I'm not given a framework 



15. I ask questions, even though there may be some risk involved 

16. I pursue answers on my own before asking questions 



1 7. I develop my own learning goals, plan for them, and achieve them 

on my own initiative 

18. I feel as though I am controlled by external factors that I cannot myself control 



19. I create my own criteria in addition to those provided by instructors 

20. I express my own perspective in thinking and communicating with others 

(do not simply disagree or agree with others) 



21. I extend my learning beyond the requirements and expectation of my courses 



2 
2 

2 

2 
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2 
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2 
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Self-Assessment 

How do I characterize (describe) my self-assessment this semester? 



22. I analyze how my behaviors affect group interaction 

23 . I recognize that my attitudes affect my work 



24. I review my work, using criteria as a basis 

25. I assess weaknesses in my own thought with the same thoroughness that I 

apply in the assessment of others 



Circle One 
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26. I can identify my own strengths and weaknesses 
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Using Feedback and Assessment Criteria 

How characteristic are the following of my learning? 



27. I compare myself to myself rather than just myself to others . . 

28. My emotional response to feedback interferes with my analysis 

of my performance 



29. I enjoy exploring assessor judgments in relation to my own analysis of 

my performance 

30. I believe assessor judgments are arbitrary and vague, giving little meaning 

for being validated 



31. I see criteria as a framework for learning that enables me to transfer 

my learning to new situations 

32. 1 want criteria to be a standard for how close I am to learning enough 

of the right answers 



33. I experience criteria as a picture of the ability I am learning or performing 

34. I know when to treat criteria as flexible and open to interpretation 



35. 1 enjoy reflecting on how one performance of mine demonstrates something 

I've noticed in another performance as well 

36. I experience feedback as affirming or rejecting who I am rather than what I do 
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37. I accept criticisms and suggestions and follow through 
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Commitment to Improve Abilities 



Are the following now characteristic of my performance? 



38. I try hard to improve my assessment performances . 

39. I seek help from others in my academic development 
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40. I actively build on strengths ^ 

41. I complete assignments that at first I do not think I can do 1 

42. I focus on using and developing my time management skills as a student 1 
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Understanding Alverno's Integrated 
Learning Process 

How characteristic are the following of my learning experiences? 

43. I need faculty to give specific examples of how various abilities relate 

to each other ^ ^ 

44. I recognize that I can consider modifying assigned procedures or directions 1 2 

45. I see that the same ability is used across my different courses 1 2 

46. I incorporate a number of abilities in approaching a problem 1 2 

47. I see abilities as tools that will be used in school person?Uife, and career 12 3 
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PERSONAL DEVELOPMENT 

Developing One's Own Perspective 
and Being Creative 

How descriptive are the following of my performance this semester? 

48. I express my ovm ideas in my work 

49. I apply my own style to my work 
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50. I create strategies to strengthen my learning experiences 

51. I use alternative learning strategies to achieve my academic goals . 

52. I expect to use my preferred learning style in all learning situations. 
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Applying Concepts 

How characteristic are the following of my learning? 



53. I attempt to apply my learning outside the classroom 

54. I believe learning is a process, i.e., that one learns how to learn 



55. I respond to and affirm relationships between theory and practice 

56. I try out new skills 



57. I need to have relationships between my everyday experiences 

and what I am learning pointed out to me 

58. lapply a newly learned ability outside of school 
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Taking Multiple Viewpoints 

How characteristic are the following of my thought and action this semester? 



59. I relate ideas, issues, from one course to another 

60. I connect ideas from coursework to other life experiences 



61. I expect faculty to tell me what is the best perspective 

62. I consider the positive implications of viewpoints that T do not prefer 
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63 . I listen to the viewpoints of others 

64. I combine my own ideas, and those of others, with those supplied 

by my course, to create new relationships 
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CAREER DEVELOPMENT 



Committing to Professionai Goals 

How characteristic are the following of my thought and action? 

65. I relate my current educational experiences to my career goals 

66. I continue to explore a variety of career options 
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67. I set new goals for my professional life 

68. I have developed professional goals . . 
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71. 



72. 



I pursue career development despite 

responsibilities, obstacles or sacrifices 

I analyze the relationship between my own abilities and 

my personal career goals 



I reflect on the broad relationships the instructor makes between what 

I am learning and what I will be doing after college 

I apply what I am learning to my professional development 
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73 . I seek cai eer information from knowledgeable sources outside the classroom 



You have completed this portion of the inventory! The next page gives you an opportunity to 
reflect on your approach to learning. 
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ST UDENT AS LEARNER INVENTORY 



Inventory Sections and Reflection Topics 



/. Committing to College Work: Alvemo instructors expect that Alvemo students will be committing 
themselves to college work. Describe what keeps you committed to college work? 

2. Becoming Independent in Learning: As the>' learn, students become more independent in tJieir learning. 
Describe a performance where you learned something because of a question you asked yourself 

3. Self-assessment: One way a student becomes more independent in learning is by developing the ability to 
self-assess or to judge her own performance. Describe what self-assessment means to you. 

4. Using Feedback and Assessment Criteria: Another way a student becomes more independent in learning is by 
developing the ability to use feedback and assessment criteria. When a student learns how to fully explore 
assessment criteria, it enables her to improve her use of feedback and to judge her performance more 
productively. Describe a time when you used feedback &om one perfomiance to improve another one. 

5. Commitment to Improve Abilities: A commitment to improving helps students to learn to self-assess and 
to use feedback. Feedback is for where the student is going — forward. Describe an ability you need to 
improve and why you want to improve it. 

6. Understanding Alvemo's Integrated Learning Process: Alvemo instructors believe that a student will be 
learning to "put it all together" (i.e., integrate everything) as she learns more about abiiity-based learning. 
Describe a time when it all "came together" for you, and what insight you had. 

7. Developing One's Own Perspective and Being Creative: Alvemo instructors expect that you will be 
developing your own unique perspective. Ability-based learning helps you to become an expert in your 
field and to work creatively. Describe what makes you most creative. 

8. Applying C .-ncepts: Alvemo instructors believe a student will apply the understanding she is developing 
to her work and personal life. Describe how you applied something you learned to another situation. 

9. Taking Multiple Viewpoints: Alvemo instructors expect that a student will leara to use more than one 
perspective in her field and that she will learn more about why multiple perspectives exist. Describe an 
opinion you disagree with and then (without endorsing it completely) describe something good about this 
altemative opinion. 

10. Committing to Professional Goals: Alvemo instructors believe that students continue to develop new 
and deepened understandings about professional careers and professional values. Students leam more 
about professional careers and more about how different and specific careers could relate to their emerging 
interests, values, and abilities. As a result, students find that future paid employment becomes meaningful 
in new ways. What does a "professional career" mean to you now? Wliat have you learned about a 
particular profession that makes it a meaningful altemative or choice for you? 
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Guidelines for writing a reflective statement about your learning 

Name Date 

On the accompanying handout, you will find descriptions of vanous sections of this Inventory. Choose one section topic 
that you want to give special thoughit to. Write a reflective paragraph or two on that topic. Wnte your reflections in the 
form of a letter to a friend with whom you would like to share your thoughts and feelings about learning. 



(add a sheet of paper ifyou like) 
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ALVERNO COLLEGE 

Milwaukee, Wl 
Office of Research and Evaluation 

Appendix B — Educator's Guide to the Student as Learner Inventory 
Context of Administration and Use in Semester 1 and 2 

The Student as Learner Inventory is administered in New Student Seminar and again in Pre- 
Professional Seminar. Instructors in these courses use the inventories as part of a guided self 
reflection on learning and careering at Alvemo. A copy of the inventory is routed immediately to 
Advisors, so that it is available for a potential conversation during registration that semester. The 
student retains a copy in her Academic and Career Resource Journal. The inventory has two 
distinct kinds of uses: (1) identifying students v^o may have learning difficulties at Alvemo and 
supporting a conversation with students about these and (2) supporting student self-reflection on 
their leaming, both by students completing the inventory and by educators validating and 
extending student self reflection. 

Identifying items That Suggest Leaming Difficulties at Alvemo 

Generally, first-year student responses indicate a self assessment of behavior or leaming 
perspective that Alvemo educators would affirm as consistent with being a beginning or 
developing leamer at Alvemo. Some student self-reports, however, seem to suggest leaming 
issues that are "pre-beginning" or "pre-thriving." For example, they may acknowledge a passive 
leaming stance or that Alverno's educational principles are "not me." While such self- 
assessments are infrequent, when they occur they suggest that the first-year student may be 
"ready " to explore the leaming issue with an Alvemo educator, so that she can better integrate 
with the leaming environment. To this purpose, the carbon copy of the Student as Leamer 
Inventory indicates the potentially "pre-thriving" responses: 

• Using the carbon copy of the inventory, review the student's self-assessment choices 
that are in bold. 

Some items have a "pre-thriving self-assessment" response identified. The "pre-thriving" 
response is either "1" or "3" and is in bold. Usually, it is the "Not Characteristic" response 
that is bolded, i.e., the "1," response.^® The more "pre-thriving" responses, the more likely the 
student may be "at risk" as a learner at Alvemo. 

• Check and count the number of responses indicating a "pre-thriving" self-assessment of 
herself. Write the total on top of the carbon copy. 

To what degree is she sharing a self-assessment that is most similar to the beginning leamer at 
Alvemo v^o is not yet thriving? If you find that she has four or more "pre-thriving" responses, she 
may be suggesting that she is having considerable difficulty connecting to the Alvemo learning 
style and may need a supportive opportuni* to "rethink" v^at it means to become an effective 
leamer at Alvemo. "Pre-thriving" responses nave been linked to a greater risk of dropping out of 
Alvemo within two years, so specifically intervening around the content of the response may be 
one targeted way to improve student retention and eventual success. 
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Giving Inventory Feedbacl< to Students 

Feed Forward For Students With Pre-Thriving Responses to Items: A strength of the 
inventory is thcit it enables students to provide a self-report on a broad range of learning 
perspectives, ideally, any "pre-thriving" self-assessment should be a topic for discussion with 
the student. Of course, the discussion would focus on the student's learning issue and her 
perspective on how she is effective in the Alverno context and would avoid unnecessary 
characterization of the student response. After you have explored the thinking behind each of 
the student's pre-thriving responses, and have found that they represent genuine issue(s) for 
the student, you might explore with the student a learning plan (e.g., suggest a future 
conversation) in relation to the self-identified area of concern. 

Feedback on Learning About Self-As-Learner Essay. Only about one-third of the beginning 
students will give even one "pre-thriving" item-response. The reflective statement that students 
complete at the end of the Student As Learner Inventory provides the most consistent data for 
exploring student insights into their own learning. A conversation with the student about her self- 
reflective statement might validate the student for her current insight, as well as to support her 
further articulation of herself as a learner. 

Postscript on Later Use By Faculty Advisors or Other Instructors 

The greater the time between the student completing the inventory and then speaking to 
someone about it, the less likely it will be a meaningful conversation. Likewise, the student 
may have changed her perspective.^^ A faculty advisor— who sees this inventory outside of its 
classroom context — might prefer to raise the learning topic without making an explicit 
reference to the student's prior response, especially if more than a semester has passed. 
However, in the context of other observations of the student, a faculty advisor might judge 
that the student's "pre-thriving" responses made much earlier in her career still suggest some 
directions for further exploration with the student on her perspective on learning at Alverno. 

We encourage faculty to experiment with uses, including further guided student self- 
reflection, perhaps through self-reflection on changes in her responses.^ From this 
perspective, a conversation later in the student's career might validate or reinforce her growth 
as a learner. 
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If faculty would like to readminister the instrument to support a conversation with a student or for another 
purpose, they may request copies of the inventory from the Office of Research and Evaluation. We are 
interested in learning from your experience, as well as in providing support where we may. Contact Glen 
Rogers (6269) or Bill Rickards (6256) in the Office of Research and Evaluation. 

The "pre-thriving responses" are not identified on the student copy, so as not to influence her initial 
Q response.. ^ ^ 
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Appendix C — Frequency of Pre-Thriving Responses^^ on 
Student As Learner Inventory Items 
Sorted by Frequency and Direction of Response 

"NOT ME" 

Rare Student Responses: "That's Not (at all) Characteristic of Me" 



Item Statement Percent 
# "Not Me" 

5. Responds to situations in which she is personally asked 

to participate 0.0% 

9. Accepts personal responsibility for learning when asked to 0.0% 

26. Knows she should improve; wants to improve; tries hard to 

improve in quality way 0.0% 

34. Explores assessor's judgments in relation to her own performance 0.0% 

12. Follows specific directions in assignments 0.5% 

52. Relates ideas, issues, from one area to another 0.5% 

60. Applies what she is learning to personal and professional life 0.5% 

3. Attends class regularly 1 .0% 

33. Sees criteria as one part of a process for learning and assessment 1 .0% 

42. Expresses her own ideas in her work 1 .0% 

1 4. Selects an appropriate framework for analyzing a work 1.1% 

1 9. Reviews her work, using criteria as a basis 1.1% 

43. Applies her own style to work 1 .6% 

57. Thinks about relationship between her own abilities and 

personal aspirations 1 .6% 

4. Shows enthusiasm for coursework 2.1% 

25. Completes assignments in weak areas; aware of weaknesses 2.1% 

45. Attempts to apply her learning outside the classroom 2.1 % 

30. Sees criteria as framework for learning that enables her 

to transfer learning 2.6% 

37. Sees that the same ability is used across her different 

courses 2.6% 

59. . "Hears" when the instructor makes broad relationships 
between what students are learning and what they will be 

doing after college 2.6% 

13. Tries a new approach to learning or problem solving when it 

has been demonstrated 3.1% 

50. Applies a newly learned ability outside of school 3.1% 

56. Pursues education/career development despite outside 

responsibilities, obstacles or sacrifices 3.1% 

38. Spontaneously incorporates a number of skills in approaching a 

problem 3.6% 

48. Tries out new skills in situations 3.6% 

10. Sees teacher as an organizer of learning opportunities 4.3% 
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Prc-thriving (or pre-beginning) responses «rc inferred based on the divergence of the student's response from one that 
would reflect integration into the Alverno learning environment. 
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"NOT ME" continued 

Infrequent Student Responses: "That's not (at all) characteristic of me" 
Item Statement Percent 

"Not Me" 

20. Examines weaknesses in lier own thought with the same rigor 

that she applies in the critique of others 5.3% * 

40. Extends her learning beyond the requirements and 

expectations of her major 6.3% 

16. Pursues answers on her own before asking questions 6.8% 

29. Sees criteria as feedback on strengths and weaknesses 7.3% 

15. Asks questions, even though there may be some risk involved 7.9% 

21 . Compares self to self, rather than just self to others 8.3% * 

6. Does more than asked for 11.5%* 

2. Does more than expected 13.0% 

"THAT'S ME" 

Rare Student Responses: "That's (really) Characteristic of Me" Responses 

Item Statement Percent 

"That's Me" 

31 . Sees assessor's judgments as arbitrary and vague, with 

little meaning for being validated 1 .0% 

P. 1 . Sees the learning of the abilities as additional work that takes 

away from her being given the course content 3.6% 

1.2. Experiences feedback as affirming or rejecting who she is, 

rather than what she does 4.7% 

Infrequent Student Responses: "That's (really) Characteristic of Me" Responses 

D.2. Places responsibility for learning on the teacher 

rather than herself 5.8% 

M.2. Sees assessor's judgments as personal opinions 9.4% * 

1 6. Feels as though she is controlled by external factors 

that she cannot herself control 9.9% 

23. Shows emotional response to feedback that interferes with 

insight into her performance 12.0% 

49. Needs to have relationships between her everyday experiences and 

what she is learning in her chosen field pointed out to her 14.1% 

Note: Based on students in New Student Seminar in the fall of 1991 . (A/ = 192) 



* Correlations conducted on infrequent pre-thriving items showed responses to these items to 
be associated with not being on campus two years later. 
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